Using active ir sensor to monitor sleep

ABSTRACT

In an implementation of the disclosed subject matter, a device may emit a first emission sequence of infrared radiation at a subject, and capture a first reflected sequence of infrared radiation reflected from the subject. The first emission sequence may be compared to the first reflected sequence, and, based on the comparison, a sequence of variations may be determined. The sequence of variations may be compared to signal pattern stored in a sleep profile for the subject. The subject may be determined to have exhibited sleep behavior based on the comparison of the sequence of variations to the signal pattern stored in the sleep profile. In response to determining the subject has exhibited sleep behavior, the device may capture a second reflected sequence of radiation reflected from the subject. A breathing rate of the subject and/or a heart rate of the subject may be determined based on the second reflected sequence.

BACKGROUND

Conventional sleep monitoring systems that measure physiologicalparameters, such as a person's heart rate or breathing rate, requiresensors to be in contact with the person. Contact sensors are oftenuncomfortable for people to wear for long periods of time. Thedisadvantages of contact sensors are magnified during sleep studies,where the presence of the sensors can influence a person's sleeppatterns. Similarly, when monitoring the sleep of an infant, contactsensors often interfere with infant's ability to sleep and otherwiseinhibit the infant's interaction with the world around them.

In addition, events detected by sleep monitoring systems, such asdangerously low breathing rates, often require an emergency response.Many homes rely on home automation systems to trigger emergencyresponses; however, conventional sleep monitoring systems do notinterface with broader home automation systems. Thus, home occupantswishing to receive health alerts via their home automation system cannotdo so.

BRIEF SUMMARY

According to an implementation of the disclosed subject matter, a devicemay emit a first emission sequence of radiation at a subject, and adevice may capture a first reflected sequence of radiation reflectedfrom the subject. The first emission sequence may be compared to thefirst reflected sequence, and, based on the comparison of the firstemission sequence to the first reflected sequence, a sequence ofvariations may be determined. The sequence of variations may be comparedto a sleep profile of the subject. The subject may be determined to haveexhibited sleep behavior based on the comparison of the sequence ofvariations to the sleep profile. In response to determining the subjecthas exhibited sleep behavior, a device may capture a second reflectedsequence of radiation reflected from the subject. A breathing rate ofthe subject and/or a heart rate of the subject may be determined basedon the second reflected sequence.

According to an implementation of the disclosed subject matter, a devicemay include a radiation emission component and a radiation capturecomponent. A processor may be in communication with the device, and theprocessor may be configured to execute instructions. The instructionsmay include emitting from a radiation emission component, a firstemission sequence of radiation at a subject; capturing at a radiationcapture component, a first reflected sequence of radiation reflectedfrom the subject; and comparing the first emission sequence to the firstreflected sequence. The instructions may include determining a sequenceof variations based on the comparison of the first emission sequence tothe first reflected sequence; comparing the sequence of variations to asleep profile of the subject; and determining, based on the comparisonof the sequence of variations to the sleep profile, that the subject hasexhibited sleep behavior. The instructions may include capturing asecond reflected sequence of radiation in response to determining thesubject has sleep behavior and determining, based on the secondreflected sequence, a breathing rate of the subject and/or a heart rateof the subject.

According to an implementation of the disclosed subject matter, anon-transitory computer readable medium may store instructions includingemitting a first emission sequence of radiation at a subject, capturinga first reflected sequence of radiation reflected from the subject, andcomparing the first emission sequence to the first reflected sequence.The instructions may include determining a sequence of variations basedon the comparison of the first emission sequence to the first reflectedsequence; comparing the sequence of variations to a sleep profile of thesubject; and determining, based on the comparison of the sequence ofvariations to the sleep profile, that the subject has exhibited sleepbehavior. The instructions may include capturing a second reflectedsequence of radiation in response to determining the subject hasexhibited sleep behavior and determining, based on the second reflectedsequence, a breathing rate of the subject and/or a heart rate of thesubject. The instructions may include comparing the breathing rateand/or the heart rate to a sleep profile of the subject; determining asleep disorder status based on the comparison of the breathing rateand/or the heart rate to the sleep profile; and providing an alert,based on the sleep disorder status, to a device associated with a user.

According to an implementation of the disclosed subject matter, a meansmay emit a first emission sequence of radiation at a subject, capture afirst reflected sequence of radiation reflected from the subject, andcompare the first emission sequence to the first reflected sequence. Ameans may determine a sequence of variations based on the comparison ofthe first emission sequence to the first reflected sequence; compare thesequence of variations to a sleep profile of the subject; and determine,based on the comparison of the sequence of variations to the sleepprofile, that the subject has exhibited a sleep behavior. A means maycapture a second reflected sequence of radiation in response todetermining the subject has exhibited sleep behavior and determine,based on the second reflected sequence, a breathing rate of the subjectand/or a heart rate of the subject.

Additional features, advantages, and embodiments of the disclosedsubject matter may be apparent from consideration of the followingdetailed description, drawings, and claims. Moreover, it is to beunderstood that both the foregoing summary and the following detaileddescription are illustrative and are intended to provide furtherexplanation without limiting the scope of the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide furtherunderstanding of the disclosed subject matter, are incorporated in andconstitute a part of this specification. The drawings also illustrateembodiments of the disclosed subject matter, and together with thedetailed description serve to explain the principles of embodiments ofthe disclosed subject matter. No attempt is made to show structuraldetails in more detail than may be necessary for a fundamentalunderstanding of the disclosed subject matter and various ways in whichit may be practiced.

FIG. 1A shows a device that includes radiation emission components andradiation capture components according to an implementation of thedisclosed subject matter.

FIG. 1B shows a device emitting radiation according to an implementationof the disclosed subject matter.

FIG. 1C shows a device emitting and capturing radiation according to animplementation of the disclosed subject matter.

FIG. 2 shows a device for emitting and capturing radiation and a subjectaccording to an implementation of the disclosed subject matter.

FIG. 3 shows a method for monitoring sleep according to animplementation of the disclosed subject matter.

FIG. 4A shows a method for determining a sleep disorder status accordingto an implementation of the disclosed subject matter.

FIG. 4B shows a sleep profile of a subject according to animplementation of the disclosed subject matter.

FIG. 5 shows a method for determining a breathing rate of a subjectaccording to an implementation of the disclosed subject matter.

FIG. 6 shows a system for monitoring sleep according to animplementation of the disclosed subject matter.

FIG. 7A shows a method for emitting radiation at a subject according toan implementation of the disclosed subject matter.

FIG. 7B shows a method for providing an alert to a device associatedwith a user based on a sleep disorder status according to animplementation of the disclosed subject matter.

FIG. 8A shows a sensor according to an implementation of the disclosedsubject matter.

FIG. 8B shows a premises according to an implementation of the disclosedsubject matter.

FIG. 9A shows a sensor according to an implementation of the disclosedsubject matter.

FIG. 9B shows a sensor according to an implementation of the disclosedsubject matter.

FIG. 10A shows networked sensors according to an implementation of thedisclosed subject matter.

FIG. 10B shows networked sensors according to an implementation of thedisclosed subject matter.

FIG. 11 shows a computing device according to an implementation of thedisclosed subject matter.

FIG. 12 shows a networked arrangement according to an implementation ofthe disclosed subject matter.

DETAILED DESCRIPTION

To address the issues previously described, devices, systems, andtechniques as disclosed herein may provide for monitoring the sleep of aperson using various measured sequences of radiation. For example, adevice that emits and captures infrared (IR) light may be located in aperson's bedroom within a premises, such as a home. Infrared ornear-infrared light will not typically wake or disturb the personbecause humans do not perceive light having infrared wavelengths. Ingeneral, implementations disclosed herein may emit a pattern or sequenceof such IR light or, more generally, radiation. By comparing lightreflected from a sleeping person to known patterns and sequences, it maybe possible to determine a status of the sleeping person, a sleepbehavior or disorder, a stage of sleep, or the like, without requiringthe sleeping person to wear or maintain a device in contact with her orher person.

In an example implementation, a device may be positioned in the bedroomin a location that enables the device to emit a first emission sequenceof radiation into a space in the bedroom that includes the bed. Thefirst emission sequence may be emitted in the pattern of an array from asingle emission component or from multiple emission components. Thedevice may capture a first reflected sequence of radiation that isreflected from the space, while a subject lays down to sleep in the bed.The first emission sequence may be compared to the first reflectedsequence, and variations between the two sequences may be determined asa sequence of variations. The sequence of variations may correspond tothe sequence of elements of the array inhabited by the subject as thesubject lays down to sleep. The sequence of variations may be comparedto a predetermined sequence of variations that is stored in a sleepprofile for the subject. Based on this comparison, it may be determinedthat the subject has exhibited sleep behavior, where the sleep behavioris behavior indicative that the subject has laid down to sleep.

After determining the subject has exhibited sleep behavior, the devicemay emit a second emission sequence of infrared radiation directed atthe subject and capture a second reflected sequence of infraredradiation reflected from the subject. The second emission sequence maybe emitted from a single emission component and may reflect from thesubject's skin, clothing, or bedding while the subject is sleeping. Thesecond reflected sequence may be analyzed using signal processingtechniques to determine dominant frequencies. Energy peaks may bedetermined in the dominant frequencies of the second reflected sequence,and these peaks may correspond to breath cycles of the subject. Based onthis determination, the breathing rate for the subject may bedetermined. Aspects of the second reflected sequence may determine tohave been reflected from the face of the subject, and these aspects maybe analyzed to detect periodic changes in wavelength. These periodicchanges in wavelength may correspond to changes in the skin tone of thesubject that are caused by variation in blood flow. The heart rate ofthe subject may be determined based on the rate energy peaks in thesewavelengths are detected.

The breathing rate data and heart rate data may be recorded andperiodically compared to sleep disorder data. The sleep disorder datamay include breathing rate values or heart rate values that correspondto sleep disorders, such as sleep apnea or dangerously low respiration.Sleep disorder data may be stored in the sleep profile for the subject.When the breathing rate or heart rate corresponds to a sleep disorderrate within a threshold amount, a sleep disorder status may bedetermined and a notice may be provided to a home automation system.This notice may trigger an alert within the home automation system suchas sending a message to another occupant of the home or sounding analarm. For example, if an infant's breathing rate falls to a dangerouslevel, an alarm may be triggered in a parent's bedroom.

Implementations of the disclosed subject matter may be partially orcompletely incorporated within home automation systems such as the“smart home environment” described in later portions of this disclosure.

A device for sleep monitoring may include radiation emission componentsand radiation capture components. The device may also includeelectronics for operation and control of the device as well asinterfacing with other components of a home automation system. Forexample, FIGS. 1A-1C show a device 100 for radiation emission andradiation capture according to an implementation of the disclosedsubject matter. The device may be a stand-alone device, may beincorporated into another device such as bedroom furniture, or coupledto a network in communication with a home automation system, sensor, orother device, such as a home monitoring hub. Any suitableelectromagnetic radiation may be emitted from the device, includingvisible white light, near-infrared radiation, or infrared radiation. Thedevice may emit radiation from emission components 110. Theimplementation depicted shows five emission components; however thedevice may include a single emission component or any number of emissioncomponents suitable for the purposes of this disclosure. The emissionand capture components may be disposed within a single housing ordevice, as shown, or the emission and capture components may be separatephysical devices that are configured to operate in tandem and/or inconjunction with another system such as a smart home system.

In FIG. 1B the emission components are depicted as infrared lightemitting diodes (LEDs), however the type of emission component may beany type of component that emits electromagnetic radiation in a mannersuitable for the purposes of this disclosure. For example the emissioncomponent may be an LED point source, a laser, or a lens-focused lightsource such as an incandescent lamp or an organic LED. In certainembodiments non-point sources may also be employed. Radiation may beemitted in a pattern 130 such as a certain arrangement of projectedpixels, an arrangement of stripes, an array that defines a set ofelements 131, and other structured formats or unstructured radiationformats. For purposes of this disclosure, a pattern may include no morethan a single element or a pattern may include multiple elements.

The device may capture radiation through capture components 120. Capturecomponents may be any suitable radiation sensor. For example the capturecomponents may be image sensors such as photodiodes, charge-coupleddevices (CCD), complementary metal-oxide-semiconductor (CMOS) devices,red green blue (RGB) imaging cameras, RGB-Depth (RGB-D) imaging cameras,infrared imaging sensors, and other components configured to detectelectromagnetic radiation. FIG. 1C shows the device emitting radiation140 from a single emission component and capturing reflected radiation150 in each image capture component. However radiation may be emittedfrom some or all emission components and captured by some or all capturecomponents. The implementation depicted shows four capture components;however the device may include a single capture component or any numberof capture components suitable for the purposes of this disclosure.

The device may include additional components sufficient to operate andcontrol the radiation emission and capture components and communicatewith other components of a home automation system, such as thecomponents of a smart home environment described below. For example, thedevice may include memory, processors, electronic circuitry, andwireless communication components such as those described with respectto FIGS. 10A-10B, FIG. 11, and FIG. 12 below.

In some implementations, the device may be positioned in a room suchthat it may emit radiation towards a space in the room where a subjectsleeps. For example, FIG. 2 shows a room 200, according to animplementation of this disclosure, where the device 100 is positionedover the bed 201. From this position, subject 210, such as a man, may bemonitored. The device may be implemented in a range of other objectswithin the room. For example, the device may be attached to orintegrated within furniture such as a bed, night stand, or dresser; alamp; overhead fan; baby's crib; a mobile over the baby's crib; alighting fixture; a ceiling, pillar, wall surface or molding; atelevision, a video monitor, an audio device or other audiovisualequipment; or any other object suitable for the purposes of thisdisclosure.

The device described above may be implemented as part of a method formonitoring the sleep of a subject. For example, FIG. 3. shows a method300 for sleep monitoring according to an implementation of the disclosedsubject matter. A first sequence of radiation may be emitted at thesubject at 310. For example, the device may emit a first emissionsequence of infrared radiation from a plurality of emitting componentstowards a space containing the subject. The sequence of radiation may beemitted in a pattern that defines an array of elements. At 320, a firstreflected sequence of radiation reflected from the subject may becaptured. For example, the device may capture the first reflected secondsequence of infrared radiation at a plurality of radiation capturecomponents housed in the device.

The first emission sequence may be compared to the first reflectedsequence at 330, and at 340 a sequence of variations between the firstemission sequence and the first reflected sequence may be determined.For example the device may emit a sequence of infrared radiation thatdefine rectangles, which may make up the elements of an array, such aselements 131 shown in FIG. 1B. When a subject goes to sleep, he may walkinto a field of view of the device, bend over, lay down on the bed, androll periodically on the bed. The subject's movements may become moreand more infrequent until he falls asleep. This course of movements mayreflect the emitted radiation as he passes through different elements ofthe array. A capture component of the device may capture the radiationreflected by the subject as he moves into a first element and detect thesubject based on a variation between the emitted pattern of radiationand captured pattern of radiation. For example, in the absence ofreflection from the subject, a line of radiation of the emitted sequenceof radiation may be anticipated to be in a predetermined location;however when the line of radiation reflects from the subject in itspath, the location may vary from this predetermined location. Based onthis variation, the presence of the subject may be detected crossinginto or out of an element of the array. This process may repeat as thesubject moves through various elements in the array in the course ofwalking into the field of view of the array, bending over, laying downon the bed, and rolling around on the bed, and falling asleep. Each ofthese variations in the emitted and reflected radiation may be combinedto make up a sequence of variations. This sequence of variations,including the time between when variations are detected, may becharacteristic of the subject's behavior when he goes to bed. Thus thesequence of variations may define a sleep behavior. Detecting this sleepbehavior may be used to distinguish between times in which the subjecthas merely gone to bed, and those in which other events take placewithin the field of view of the device, such as when the subject iswatching television or when a pet jumps onto the bed.

Depth within the array may also be determined, resulting in a variationthat represents a three dimensional location of the subject. Forexample, a depth of the subject may be determined through modulated timeof flight sensing techniques that detect a variation in a the phase of acarrier signal between the first emission sequence of radiation and thefirst reflected sequence of radiation. In this way the distance betweenthe device and the subject from whom the radiation is reflected may bedetermined and combined into a three dimensional location of thesubject. Each successive three dimensional location may be combined asabove to make up a sequence of variations corresponding to threedimensional locations of the subject.

Various techniques, such as structured light techniques, stereotechniques, and time-of-flight sensing, may be employed when determiningthe location of the subject. For example, fixed or programmablestructured light techniques may be employed to detect variations in apattern of radiation, such as the dimensional spreading, geometricalskewing, or depth of the pattern's elements, in order to determineinformation about an object. An example of such a technique is providedin Geng, Jason, “Structured-light 3D surface imaging: a tutorial”,Advances in Optics and Photonics 3, no. 2 (2011): 128-160. In addition,stereo techniques may be employed to detect a variation between thelocation of an aspect of a pattern of radiation captured in a firstcapture component and the location of the aspect in a second capturecomponent. This variation may be used to determine location and depthinformation of the object from which the pattern is reflected. Anexample of such a technique is provided in Alhwarin, Faraj, et al., “IRstereo kinect: improving depth images by combining structured light withIR stereo”, PRICAI 2014: Trends in Artificial Intelligence, pp. 409-421,Springer International Publishing, 2014. As another example, atime-of-flight variation may be measured between a pulse emission of apattern of radiation and the captured reflection of that pattern ofradiation, or a time-of-flight variation may be measured by determiningthe phase shift between an emitted pattern of radiation modulated by acontinuous wave and the captured reflection of that pattern ofradiation. Time-of-flight variations such as these may be used todetermine location and depth information of an object. An example ofsuch a technique is provided in Zhu, Jiejie et al., “Fusion oftime-of-flight depth and stereo for high accuracy depth maps”, ComputerVision and Pattern Recognition, 2008, CVPR 2008. IEEE Conference on, pp.1-8, IEEE, 2008.

Continuing the description of FIG. 3, the sequence of variations may becompared to a sleep profile of the subject at 350, and based on thiscomparison, it may be determined that the subject is exhibiting sleepbehavior at 360. For example, a sequence of variations corresponding tothe subject going to bed may be stored in a sleep profile of thesubject. The determined sequence of variations may be compared to thestored sequence of variations and the discrepancy may be measured. Forexample, the stored sequence of variations may correspond to a sequenceof coordinates of the subject has moved through in the past when he goesto sleep. The determined sequence of variations may correspond to asequence of coordinates of the subject as he is going to sleep in thepresent. The sequences may be compared and the discrepancies between theeach set of coordinates quantified. This quantity may be compared to athreshold value. If the quantity is below the threshold then it may bedetermined the subject has laid down to go to sleep.

The threshold for detecting sleep behavior may be determinedempirically. For example the device may collect data from the subject asthe subject goes to sleep over a period of time and determine an averagestatistical variance in the determined sequences of variations. Thisvariance may serve as the threshold value. The threshold also may be aconfiguration of the subject matter of this disclosure. For example auser may specify the threshold value based on the degree of sensitivitydesired.

Once a subject has been determined to be exhibiting sleep behavior, thesubject may be monitored, and a breathing rate or heart rate of thesubject may be determined. For example, at 370 a second reflectedsequence of radiation reflected from the subject may be captured. Thesecond reflected sequence may be reflected from radiation thatoriginated as some or all of the first emission sequence, or the secondreflected sequence may be reflected from a distinct sequence ofradiation emitted from the device. At 380 based on the second reflectedsequence, a breathing rate of the subject and a heart rate of thesubject may be determined. The second reflected sequence may beradiation reflected from a single component, such as an LED pointsource, or it may be radiation reflected from multiple components.

The breathing rate of the subject may be determined according to varioustechniques. For example, body surface movements in the chest and abdomenarea due to respiration may alter the distance between the subject andthe device capturing reflected radiation, as well as the geometry of thereflecting surfaces of the subject. For example, the subject may inhaleand the subject's chest may expand. This expansion may shorten thedistance between the radiation capture component and the subject, or theexpansion may alter the angle of bedding surface covering the subject'schest and thus increase or decrease the surface area reflectingradiation. Movements such as this may increase or decrease parameters ofthe reflected radiation such as frequency, wavelength, and intensity.

The variations in the second reflected sequence of radiation may bedetected and may indicate respiration events. For example, the amount ofenergy reflected may vary periodically in accordance with the breathingrate due to movements in the subject's chest and abdomen. There may becharacteristic peaks or troughs in reflected energy. The breathing rateof the subject may be determined by detecting these variations. Examplesof techniques for determining breathing rate may be found in Boccanfuso,Laura, and Jason M. O'Kane. “Remote measurement of breathing rate inreal time using a high precision, single-point infrared temperaturesensors”, Biomedical Robotics and Biomnechatronics (BioRob), 2012 4thIEEE RAS & EA IBS International Conference on, pp. 1704-1709. IEEE,2012. Temperature changes in the skin around the nose and mouth of thesubject may also be detected based on reflected infrared radiation andused as a basis for determining breathing rates. Examples of such facialtemperature techniques may be found in Zhao, Fang, Meng Li, Yi Qian, andJoe Z. Tsien. “Remote measurements of heart and respiration rates fortelemedicine.” PloS one 8, no. 10 (2013): e71384.

Before determining the breathing rate, the degree of motion of thesubject may be determined. In general the deeper the sleep stage of thesubject the less movement the subject will exhibit, and thus the lessgeneral bodily movement signals may obscure breathing rate signals.Thus, in some implementations, breathing rates may be easier todetermine when the subject is in stage 4 sleep than when the subject isin and out of stage 1 sleep. The degree of bodily movement may bedetermined by analyzing the “noisiness” of the second reflected sequenceradiation. If, for example, the second reflected sequence varies morethan a threshold amount, then determinations of the breathing rate maybe postponed until the subject has entered a deeper sleep stage.

The heart rate of the subject may be determined by detecting variationsin the wavelength or related parameters (for example: frequency, energy,intensity) of light reflected from the face or other skin surfaces ofthe subject. For example, a subject's heart rate may be correlated withthe rate blood flows through the subject's veins. Changes in bloodvolume or oxygenation levels in blood may cause changes in skin tonecolor due to veins beneath the skin. These changes in skin tone mayincrease the amount of energy in certain wavelengths or frequencies inreflected light. Variations in wavelength or related parameters of thesecond reflected sequence of radiation may be detected using signalprocessing techniques. The subject's heart rate may be determined basedon the frequency of these parameters. Examples of these techniques maybe found in Boccanfuso, Laura, and Jason M. O'Kane mentioned above.

Various techniques may be employed to perform the signal processingtasks discussed herein. For example, statistical techniques, Markovmodels, state-machine approaches, machine learning techniques,probabilistic models, as well as any other signal processing techniquessuitable for the purposes of this disclosure may be employed.Characteristic signal patterns used for determining the phenomenadisclosed herein may be made up of distributions of frequencies,wavelengths, energies, intensities, power, or related parameterssuitable for the subject matter of this disclosure. Variations may beperiodic features within a distribution or between two or moredistributions. Variations may be determined by, for example, the signalprocessing techniques discussed above, as well as other techniquessuitable for the purposes of this disclosure.

Sleep disorders may be detected based on the determined breathing rateand heart rate, as well as other signal data. For example, FIG. 4Adepicts a method 400 for determining a sleep disorder status accordingto an implementation of the disclosed subject matter. At 410 thebreathing rate and heart rate of the subject may be compared to disorderrates stored in the sleep profile for the subject, and at 420 a sleepdisorder status may be determined based on the comparison. A sleepdisorder status may be any status of a device or system that isindicative of a sleep disorder exhibited by the subject. For example, asleep disorder status may be data occupying a field corresponding to asleep disorder in an application executing on a device or system. Asleep disorder status may include various types of indicators. Forexample, the sleep disorder status may indicate the particular sleepdisorder detected or merely the logical existence of a sleep disorder.

A sleep disorder may be an abnormal heart rate, an abnormal breathingrate, or abnormal movements of the subject as compared to baseline ratesand tolerances in a subject's sleep profile. A sleep disorder may alsoinclude higher level diagnoses. For example, if a subject is exhibitinga slow and irregular breathing rate, a sleep disorder may be sleepapnea. Subject-specific sleep disorders may be specified within thesubject's sleep profile. For example, a sleep disorder may be specifiedas exhibiting a heart rate within an otherwise generally healthy rangeif the subject is also known to have a heart condition that rendersexperiencing such heart rates dangerous. As another example, a sleepdisorder may be specified that corresponds to an awake child based onthe child moving around when the child should be sleeping. In general asleep disorder may be any combination of physiological parameters andmovements specified as a sleep disorder within a subject's sleepprofile.

An example of a sleep profile according to an implementation of thedisclosure is shown in FIG. 4B. As show, the profile may store signalpatterns A1-A4 made up of a sequence of variations corresponding to thesubject exhibiting sleep behavior. These signal patterns may bedeveloped empirically, for example by capturing reflected radiation overa series of events when the subject goes to bed. The multiple variantsA1-A4 may correspond to different sleep behaviors exhibited. Forexample, the subject may have a signal pattern stored for when thesubject goes to bed on the left side of the bed, a signal pattern forwhen the subject goes bed on the right side of the bed, a signal patternfor when the subject's spouse goes to bed with the subject, a signalpattern for when the subject reads a book before going to sleep, asignal pattern for when the subject watches television and then goes tosleep, and so forth. A set of variants may be desirable so thatbehaviors leading up to the subject exhibiting sleep behavior do notobscure legitimate opportunities to monitor physiological parameters.

The sleep profile may also include, for example, values of the normalbreathing rate B for the subject and normal heart rate C for thesubject, These may be determined empirically through a series ofobservations of the subject with the device, or they may be know throughother diagnostic methods or received as input from a user. Accompanyingthe normal breathing rate and normal heart rate data may be thresholdvalues or tolerances. These values may be quantities or ranges thatcorrespond, for example, to empirically determined average variances inthe heart rate and breathing rate for the subject; they may bepercentages, or they may be configuration values specified for thesubject based upon the subject's circumstances. There may also bevariants of normal breathing rate B and heart rate C. For example, thesubject may exhibit a certain breathing rate within the first hour afterthe subject goes to bed, but this breathing rate may slow significantlylater in the sleep cycle as the subject experiences deeper sleep stages.

The sleep profile may also include additional signal patterns,quantities, ranges, tolerances, and threshold values specified for thesubject. For example, a characteristic signal may be determined thatcorresponds to the various sleep stages D-I the subject experiences.Thus, based on comparing reflected sequences of radiation to thesecharacteristic sleep stage signal patterns, the particular sleep stagethe subject is exhibiting may be determined. This may be desirable fordetermining whether the subject is completing full, healthy sleepcycles, or whether, for example, the subject is not reaching Rapid EyeMovement (REM) sleep. In another example, a signal may be determinedthat is characteristic of the subject experiencing sleep apnea. Thus, bycomparing reflected sequences of radiation to the characteristic sleepapnea signal pattern, it may be determined that the subject isexperiencing the sleep disorder of sleep apnea. Determinations based onsignal comparisons may also be combined with determinations based onbreathing rate determinations to allow for a more robust diagnosis ofsleep apnea. As another example, a signal may be determined that ischaracteristic of the subject moving around after having already gone tosleep. For example, a characteristic signal may be determined based onsequence of variations captured when a child moves through variouselements in an emitted array that is projected towards the child's bed.

Returning to FIG. 4A, at 430 a notice may be provided to a homeautomation system based on the sleep disorder status. As discussedabove, the notice may trigger an alert or provide a data stream to otheroccupants of the home, such as through a home automation system. Furtherdiscussion of the notice aspects of this disclosure is provided below.

In some circumstances, it may be desirable to determine the subject hasentered a sleep stage before determining the breathing rate of thesubject. For example, as mentioned above, it may be advantageous todetermine a subject has entered a sleep stage where the subject will notbe expected to exhibit significant bodily motion. For example, thesubject may “toss and turn” or otherwise adjust the subject's positionwhile sleeping, or the subject may exhibit involuntary movements such asRestless Legs Syndrome, Periodic Limb Movement Disorder, or forms ofMyoclonus that adjust the position of the torso. These movements mayalter the radiation reflected from the torso of the subject. Theseadditional alterations may introduce nonperiodic variations that distortor mask a signal of periodic phenomena from which a breathing rate maybe determined.

In an example, FIG. 5 depicts a method 500 for determining a breathingrate of a subject according to an implementation of the disclosedsubject matter. At 510 a sequence of radiation reflected from thesubject is captured and a first variation in a quantity of radiation inthe reflected sequence is detected. For example, a signal pattern madeup of a series of repeating energy peaks may be detected within thereflected sequence. At 520 the subject may be determined to have enteredinto a sleep stage. For example the detected signal pattern may becompared to a signal pattern corresponding to a sleep stage stored inthe sleep profile for the subject. Based on a degree of similaritybetween the detected pattern and stored pattern, the subject may bedetermined to have entered into a particular sleep stage.

A sleep stage may be a recognized sleep stage, such as those set forthin Iber C, Ancoli-Israel S, Chesson A, Quan S F, eds. The AASM Manualfor the Scoring of Sleep and Associated Events: Rules, Terminology, andTechnical Specifications, 1st ed. Westchester, Ill.: American Academy ofSleep Medicine, 2007. For example, a sleep stage may be any of thecharacteristic periods of time over the course of a subject's sleepcycle, such as the wakefulness stage, stages 1-3, and REM sleep (stage 3is sometimes recognized as split into stage 3 and stage 4/near-REMsleep). It may be advantageous to determine a subject has entered into aparticular recognized sleep stage because the sleep stage may beassociated with low bodily movement. For example, by determining that asubject has entered stage 3, it may be determined that the subject is inan advantageous condition for measuring the breathing rate of thesubject. Thus by detecting a particular sleep stage, a particular timeframe may also be determined when it may be advantageous to determine abreathing rate.

Signal patterns correlated with a sleep stage of a subject may bedetermined for a subject and stored in the subject's sleep profile forfuture use. For example, a sleep stage may be detected using methodssuch as those set forth in the AASM Manual, such as EEG or similartechniques. While these behaviors are being detected, sequences ofradiation may be emitted, reflected, and captured by the a radiationcapture component of the device disclosed herein. These capturedsequences may be analyzed and periodic phenomena may be identified usingsignal analysis techniques.

For example, periodic phenomena correlated to a specific sleep stage maybe identified by detecting dominant energy peaks or related parametersin the signal of the reflected sequence of radiation. These periodicvariations may be stored in a sleep profile for the subject. Whendetermining whether the subject has entered a particular sleep stage,sequences of radiation may be captured and compared to these storedsignal patterns. Difference between the detected signal pattern and thestored signal pattern may be measured. For example, the differencebetween a quantity in an energy peak of the captured pattern and thestored pattern may be determined. Other variations may also be detected,such as differences in signal frequency, periods of time when peaks aremeasured, or the intensity of captured radiation. The sum of differencesin energy peak values may be calculated over a period of time anddivided by a number of energy peaks to result in an average differencevalue. This sum may be compared to a threshold value. For example thethreshold value may be a standard deviation in energy peak values in thestored signal pattern. Thus for example, if the average difference valueis less than the threshold, then the subject may be determined to be inthe sleep stage correlated to the stored signal pattern. Otherthresholds may also be employed such as the standard deviations ofparticular populations of subjects, or a threshold may be selected as aconfiguration for the device. For example, a threshold may be a degreeof difference, such as a percentage that the sum of the energy ofmeasured peaks differs from the sum of the energy of the peaks in astored signal pattern. Other thresholds suitable for the purposes ofthis disclosure may also be employed.

A sleep stage may also be a selected configuration rather than aparticular recognized sleep stage such as those specified in the AASMguidelines. For example, rather than determining a subject is in aparticular sleep stage using the techniques described above and thencorrelating the sleep stage to a signal pattern in reflected radiation,general periodicity in captured radiation may be detected and used as abasis to trigger determination of a breathing rate. This generalperiodicity may be deemed to be a sleep stage.

For example, a sequence of reflected radiation may be captured and itmay be determined that the captured sequence exhibits periodic phenomenawithin a certain time window. This periodic phenomena may be measured tooccur for a period of time, such as 180 seconds. In response tomeasuring the periodic phenomena for this period of time, it may bedetermined that it is likely that the periodic phenomena will continuefor a sufficient time in the future such that a breathing rate may bemeasured. For example, characteristic energy peaks in a signal may bedetermined to occur in a window every 3 to 5 seconds when the subject issleeping. This signal pattern may be detected for 180 seconds. Inresponse to detecting this signal pattern for 180 seconds, furthersignal processing may be applied to the sequence of reflected radiationto determine a breathing rate. For example, signal processing techniquesmay be employed to detect dominant energy peaks within the 3 to 5 secondsignal window and these dominant peaks may be used as markers formeasuring the breathing rate of the subject. In this way, generalperiodic phenomena may be detected, and the detection of this generalperiodicity for a period of time may be determined to be a sleep stageof the subject. Detecting the sleep stage of the subject may be used asa trigger from which to determine a breathing rate of the subject.

Thus, as shown in FIG. 5, if the subject has entered into a particularsleep stage, a second variation in a quantity of radiation in thereflected sequence may be detected at 530, and the breathing rate of thesubject may be determined at 540, such as in accordance with thetechniques discussed above. If the subject is determined at 520 to be ina lower sleep stage, such as stage one, the subject may exhibit moremovement than is optimal for determining the breathing rate. Thus, ifthe sleep stage is not determined to be appropriate, furtherdeterminations of the breathing rate be postponed. Whether a particularsleep stage is sufficient for determining a breathing rate may bespecific to the subject. Thus conditions optimal for determiningbreathing rates for a subject may be determined and implemented as auser-specific configuration.

Implementations of the disclosed subject matter may be embodied insystems such as that disclosed in FIG. 6. System 600 may include device610 having a radiation emission component and a radiation capturecomponent, and a processor 620 in communication with the device, forexample over network 630. The processor may be configured to executeinstructions including emitting, from the radiation emission component,a first emission sequence of radiation at a subject and capturing, atthe radiation capture component, a first reflected sequence of radiationreflected from the subject. The instructions may include comparing thefirst emission sequence to the first reflected sequence; determining asequence of variations based on the comparison of the first emissionsequence to the first reflected sequence; comparing the sequence ofvariations to a sleep profile of the subject; and determining, based onthe comparison of the sequence of variations to the sleep profile, thatthe subject has exhibited sleep behavior. The instructions may furtherinclude, in response to determining the subject has exhibited sleepbehavior, capturing, at the radiation capture component, a secondreflected sequence of radiation and determining, based on the secondreflected sequence, a breathing rate of the subject and/or a heart rateof the subject. Instructions, for executing any of the methods orprocesses disclosed herein, such as those discussed above, may bestored, for example, in a non-transitory computer readable storagemedium.

Other components of system 600 may include, for example, room profile640, premises data 650, and user data 660, all of which may be store indatabase implemented in storage devices. Home monitoring hub 670 mayalso be included in the system. These components may be part of a homeautomation system that may make up or be part of the smart homeenvironment described below. For example, the sleep monitoringtechniques described herein may be part of a suit of capabilitiesintegrated into a given room, such as automated lighting systems,automated heating and cooling systems, security systems, and audiovisualsystems. Each room may have configurations for these various settings,which may be stored and managed within a room profile for the room.Similarly, each user of a home may have certain user specificconfigurations for each system of a smart home environment.Configurations for certain subjects may be stored and managed as part ofa set of user data for each occupant of the home. Premises data mayinclude data from sensors and data sets associated with the home.

In an example implementation, a sleep monitoring system may receivepremises data that indicates the cooling system is malfunctioning. Thismay result in significantly higher temperatures in a room than those forwhich the room's sleep profiles were configured. As a result, heart ratedeterminations may be affected because skin tones may be warmer and thusreflect higher energy signals. The sleep monitoring system may accountfor this environmental change automatically by filtering certainwavelengths from captured signals or reducing energy peaks by acommensurate amount in order to maintain accurate measurements. The homemonitoring hub may coordinate these various system as well as providemanagement of distributed processing and data storage requirements thatsupport the smart home environment.

Implementations disclosed herein may include those in which a singledevice may emit radiation and capture the radiation. For example, FIG.7A shows method 700 according to an implementation of the disclosedsubject matter, where at 710 the second reflected sequence of radiationcaptured by the device includes radiation previously emitted from thedevice and reflected from the subject. This previous emission ofradiation may be an emitted sequence of radiation that is distinct fromthe first emitted sequence of radiation, or it may be part of the firstemitted sequence of radiation. In other implementations, one device mayemit radiation and another device may capture the radiation, multipledevices may emit radiation and one device may capture the radiation, orone device may emit radiation and multiple devices may capture theradiation. Similarly, each such device may only emit, only capture, orboth capture and emit radiation. For example, in some implementations itmay be advantageous to have a capturing device in closer proximity tothe face of a subject to increase the signal quality gathered from theskin. The emitting device may be positioned further from the subject inorder to capture a wider range of sleeping behavior. In general, unlessexplicitly indicated otherwise herein, any combination of emitting andcapturing devices may be used.

Implementations of the disclosed subject matter may provide a notice toa home automation system based on determining a sleep disorder status ofa subject. For example, FIG. 7B shows method 720 for providing an alertto a user. At 730 a sleep disorder status for a user may be determined.For example, the subject may be an infant and it may be determined thatthe subject's breathing rate has fallen below a threshold value. Inresponse to this determination, an alert may be provided to a deviceassociated with a user of the disclosed subject matter at 740. Forexample, the infant's parent's mobile device may receive an alert. Inanother example, emergency personnel may be automatically alerted andother components of the smart home environment may be activated. Forexample, the infant's parent may receive an alert on her mobile device,a home healthcare provider or an emergency responder may be alerted, anda camera in the infant's room may be activated and begin streaming avideo feed to the parent's device.

The methods, systems, and devices set forth in the subject matter ofthis disclosure may be in communication with other methods, systems, anddevices throughout a premises. Combined these systems, methods, anddevices may make up the greater smart home environment for the premises.Additional aspects of the smart home environment and related componentsare discussed in the following portions of this disclosure.

In general, a “sensor” as disclosed herein may include multiple sensorsor sub-sensors, such as a position sensor that includes both a globalpositioning sensor (GPS) as well as a wireless network sensor. Thiscombination may provide data that can be correlated with known wirelessnetworks to obtain location information. Multiple sensors may bearranged in a single physical housing, such as where a single deviceincludes movement, temperature, magnetic, and/or other sensors, as wellas the devices discussed in earlier portions of this disclosure. Such ahousing also may be referred to as a sensor or a sensor device. Forclarity, sensors are described with respect to the particular functionsthey perform and/or the particular physical hardware used, when suchspecification is necessary for understanding of the embodimentsdisclosed herein.

A sensor may include hardware in addition to the specific physicalsensor that obtains information about the environment. FIG. 8A shows anexample sensor as disclosed herein. The sensor 810 may include anenvironmental sensor 820, such as a temperature sensor, smoke sensor,carbon monoxide sensor, motion sensor, accelerometer, proximity sensor,passive infrared (PIR) sensor, magnetic field sensor, radio frequency(RF) sensor, light sensor, such as any of the devices discussed inearlier portions of this disclosure, humidity sensor, pressure sensor,microphone, or any other suitable environmental sensor, that obtains acorresponding type of information about the environment in which thesensor 810 is located. A processor 830 may receive and analyze dataobtained by the sensor 810, control operation of other components of thesensor 810, and process communication between the sensor and otherdevices. The processor 830 may execute instructions stored on acomputer-readable memory 840. The memory 840 or another memory in thesensor 810 may also store environmental data obtained by the sensor 810.A communication interface 850, such as a Wi-Fi or other wirelessinterface, Ethernet or other local network interface, or the like mayallow for communication by the sensor 810 with other devices. A userinterface (UI) 860 may provide information and/or receive input from auser of the sensor. The UI 860 may include, for example, a speaker tooutput an audible alarm when an event is detected by the sensor 810.Alternatively, or in addition, the UI 860 may include a light to beactivated when an event is detected by the sensor 810. The userinterface may be relatively minimal, such as a liquid crystal display(LCD), LED display, or limited-output display, or it may be afull-featured interface such as a touchscreen. Components within thesensor 810 may transmit and receive information to and from one anothervia an internal bus or other mechanism as will be readily understood byone of skill in the art. One or more components may be implemented in asingle physical arrangement, such as where multiple components areimplemented on a single integrated circuit. Sensors as disclosed hereinmay include other components, and/or may not include all of theillustrative components shown.

As an example of the implementation of sensors within a premises FIG. 8Bdepicts, one or more sensors implemented in a home premises 870 as partof a smart home environment. The smart home environment may includemultiple types of home automation devices, such as one or moreintelligent, multi-sensing, network-connected thermostats 872, one ormore intelligent, multi-sensing, network-connected poisonous gasdetection units 873, one or more intelligent, multi-sensing,network-connected entry detection units 875, and one or morenetwork-connected door handles 876.

In some configurations, two or more sensors may generate data that canbe used by a processor of a system to generate a response and/or infer astate of the environment. For example, an ambient light sensor in a roommay determine that the room is dark (e.g., less than 60 lux). Amicrophone in the room may detect a sound above a set threshold, such as60 dB. The system processor may determine, based on the data generatedby both sensors, that it should activate one or more lights in the room.In the event the processor only received data from the ambient lightsensor, the system may not have any basis to alter the state of thelighting in the room. Similarly, if the processor only received datafrom the microphone, the system may lack sufficient data to determinewhether activating the lights in the room is necessary, for example,during the day the room may already be bright or during the night thelights may already be on. As another example, two or more sensors maycommunicate with one another. Thus, data generated by multiple sensorssimultaneously or nearly simultaneously may be used to determine a stateof an environment and, based on the determined state, generate aresponse.

As another example, a system may employ a magnetometer affixed to adoorjamb and a magnet affixed to the door. When the door is closed, themagnetometer may detect the magnetic field emanating from the magnet. Ifthe door is opened, the increased distance may cause the magnetic fieldnear the magnetometer to be too weak to be detected by the magnetometer.If the system is activated, it may interpret such non-detection as thedoor being ajar or open. In some configurations, a separate sensor or asensor integrated into one or more of the magnetometer and/or magnet maybe incorporated to provide data regarding the status of the door. Forexample, an accelerometer and/or a compass may be affixed to the doorand indicate the status of the door and/or augment the data provided bythe magnetometer. FIG. 9A shows a schematic representation of an exampleof a door that opens by a hinge mechanism 910. In the first position920, the door is closed and the compass 980 may indicate a firstdirection. The door may be opened at a variety of positions as shown930, 940, and 950. The fourth position 950 may represent the maximumamount the door can be opened. Based on the compass 980 readings, theposition of the door may be determined and/or distinguished morespecifically than merely open or closed. In the second position 930, forexample, the door may not be far enough apart for a subject to enter thehome. A compass or similar sensor may be used in conjunction with amagnet, such as to more precisely determine a distance from the magnet,or it may be used alone and provide environmental information based onthe ambient magnetic field, as with a conventional compass.

FIG. 9B shows a compass 980 in two different positions, 920 and 940,from FIG. 9A. In the first position 920, the compass detects a firstdirection 960. The compass's direction is indicated as 970 and it may bea known distance from a particular location. For example, when affixedto a door, the compass may automatically determine the distance from thedoorjamb or a user may input a distance from the door jamb. The distance960 representing how far away from the doorjamb the door is may becomputed by a variety of trigonometric formulas. In the first position920, the door is indicated as not being separate from the door jamb(i.e., closed). Although features 960 and 970 are shown as distinct inFIG. 9B, they may overlap entirely. In the second position 940, thedistance 990 between the doorjamb and the door may indicate that thedoor has been opened wide enough that a subject may enter. Thus, thesensors may be integrated into a home system, mesh network, or work incombination with other sensors positioned in and/or around anenvironment.

In some configurations, an accelerometer may be employed to indicate howquickly the door is moving. For example, the door may be lightly movingdue to a breeze. This may be contrasted with a rapid movement due to asubject swinging the door open. The data generated by the compass,accelerometer, and/or magnetometer may be analyzed and/or provided to acentral system such as a controller 1030 and/or remote system 1040depicted in FIG. 10A. The data may be analyzed to learn a user behavior,an environment state, and/or as a component of a smart home system.While the above example is described in the context of a door, a subjecthaving ordinary skill in the art will appreciate the applicability ofthe disclosed subject matter to other implementations such as a window,garage door, fireplace doors, vehicle windows/doors, faucet positions(e.g., an outdoor spigot), a gate, seating position, other openings,etc.

The data collected from one or more sensors may be used to determine thephysical status and/or occupancy status of a premises. For example,open/close sensors such as door sensors as described with respect toFIGS. 9A and 9B may be used to determine that an unknown subject hasentered the premises. The system may first determine that a subject hasentered the premises due to sensors detecting a door opening and closingin a time span previously determined to be consistent with a subjectentering or leaving the premises. The system next may identify thesubject as “unknown” due to the absence of a smartphone, key fob,wearable device, or other device typically used to identify occupants ofthe premises. Continuing the example, sensor data may be receivedindicating that a valuable item within the premises has been moved, orthat a component of the smart home environment associated with securityfunctions such as a controller disclosed herein, has been moved ordamaged. Such sensor data may be received, for example, from a sensorattached to or otherwise associated with the valuable item, from thesmart home component itself, or from one or more other sensors withinthe smart home environment. In response, the system may generate analert indicating that an unknown subject has entered the premises and/orthat the item or component has been moved or damaged. The system mayfurther determine that an occupant of the home is close by but notpresent in the premises, for example based upon a Wi-Fi signal receivedfrom the occupant's smartphone, but an absence of near-field or othershort-range communication from the same smartphone. In this case, thesystem may be configured to send the alert to the occupant's smartphone,such as via SMS, email, or other communication. As another example, thesystem may determine that the premises is already in an “away” state andthat no occupants are nearby or expected to return in the near future.In this case, the system may be configured to send the alert to a locallaw enforcement agency, such as via email, SMS, recorded phone call, orthe like.

Data generated by one or more sensors may indicate patterns in thebehavior of one or more users and/or an environment state over time, andthus may be used to “learn” such characteristics. For example, sequencesof patterns of radiation may be collected by a capture component of adevice in a room of a premises and used as a basis to learn objectcharacteristics of a user, pets, furniture, plants, and other objects inthe room. These object characteristics may make up a room profile of theroom and may be used to make determinations about objects detected inthe room.

In another example, data generated by an ambient light sensor in a roomof a house and the time of day may be stored in a local or remotestorage medium with the permission of an end user. A processor incommunication with the storage medium may compute a behavior based onthe data generated by the light sensor. The light sensor data mayindicate that the amount of light detected increases until anapproximate time or time period, such as 3:30 pm, and then declinesuntil another approximate time or time period, such as 5:30 pm, at whichpoint there is an abrupt increase in the amount of light detected. Inmany cases, the amount of light detected after the second time periodmay be either below a dark level of light (e.g., under or equal to 60lux) or bright (e.g., equal to or above 400 lux). In this example, thedata may indicate that after 5:30 pm, an occupant is turning on/off alight as the occupant of the room in which the sensor is locatedenters/leaves the room. At other times, the light sensor data mayindicate that no lights are turned on/off in the room. The system,therefore, may learn occupants' patterns of turning on and off lights,and may generate a response to the learned behavior. For example, at5:30 pm, a smart home environment or other sensor network mayautomatically activate the lights in the room if it detects an occupantin proximity to the home. In some embodiments, such behavior patternsmay be verified using other sensors. Continuing the example, userbehavior regarding specific lights may be verified and/or furtherrefined based upon states of, or data gathered by, smart switches,outlets, lamps, and the like.

Such learning behavior may be implemented in accordance with thetechniques disclosed herein. For example, a smart home environment asdisclosed herein may be configured to learn appropriate notices togenerate or other actions to take in response to a determination that anotice should be generated, and/or appropriate recipients of aparticular notice or type of notice. As a specific example, a smart homeenvironment may determine that after a notice has been sent to a firstoccupant of the smart home premises indicating that a window in a roomhas been left open, a second occupant is always detected in the roomwithin a threshold time period, and the window is closed shortlythereafter. After making such a determination, in future occurrences thenotice may be sent to the second occupant or to both occupants for thepurposes of improving the efficacy of the notice. In an embodiment, such“learned” behaviors may be reviewed, overridden, modified, or the likeby a user of the system, such as via a computer-provided interface to asmart home environment as disclosed herein.

Sensors as disclosed herein may operate within a communication network,such as a conventional wireless network, and/or a sensor-specificnetwork through which sensors may communicate with one another and/orwith dedicated other devices. In some configurations one or more sensorsmay provide information to one or more other sensors, to a centralcontroller, or to any other device capable of communicating on a networkwith the one or more sensors. A central controller may be general- orspecial-purpose. For example, one type of central controller is a homeautomation network that collects and analyzes data from one or moresensors within the home. Another example of a central controller is aspecial-purpose controller that is dedicated to a subset of functions,such as a security controller that collects and analyzes sensor dataprimarily or exclusively as it relates to various securityconsiderations for a location. A central controller may be locatedlocally with respect to the sensors with which it communicates and fromwhich it obtains sensor data, such as in the case where it is positionedwithin a home that includes a home automation and/or sensor network.Alternatively or in addition, a central controller as disclosed hereinmay be remote from the sensors, such as where the central controller isimplemented as a cloud-based system that communicates with multiplesensors, which may be located at multiple locations and may be local orremote with respect to one another.

FIG. 10A shows an example of a sensor network as disclosed herein, whichmay be implemented over any suitable wired and/or wireless communicationnetworks. One or more sensors 1010 and 1020 may communicate via a localnetwork 1000, such as a Wi-Fi or other suitable network, with each otherand/or with a controller 1030. The controller may be a general- orspecial-purpose computer. The controller may, for example, receive,aggregate, and/or analyze environmental information received from thesensors 1010 and 1020. The sensors 1010 and 1020 and the controller 1030may be located locally to one another, such as within a single dwelling,office space, building, room, or the like, or they may be remote fromeach other, such as where the controller 1030 is implemented in a remotesystem 1040 such as a cloud-based reporting and/or analysis system.Alternatively or in addition, sensors may communicate directly with aremote system 1040. The remote system 1040 may, for example, aggregatedata from multiple locations, provide instruction, software updates,and/or aggregated data to a controller 1030 and/or sensors 1010, 1020.

The devices of the disclosed subject matter may be communicativelyconnected via the network 1000, which may be a mesh-type network such asThread, which provides network architecture and/or protocols for devicesto communicate with one another. Typical home networks may have a singledevice point of communications. Such networks may be prone to failure,such that devices of the network cannot communicate with one anotherwhen the single device point does not operate normally. The mesh-typenetwork of Thread, which may be used in methods and systems of thedisclosed subject matter may avoid communication using a single device.That is, in the mesh-type network, such as network 1000, there is nosingle point of communication that may fail so as to prohibit devicescoupled to the network from communicating with one another.

The communication and network protocols used by the devicescommunicatively coupled to the network 1000 may provide securecommunications, minimize the amount of power used (i.e., be powerefficient), and support a wide variety of devices and/or products in ahome, such as appliances, access control, climate control, energymanagement, lighting, safety, and security. For example, the protocolssupported by the network and the devices connected thereto may have anopen protocol which may carry IPv6 natively.

The Thread network, such as network 1000, may be easy to set up andsecure to use. The network 1000 may use an authentication scheme, suchas AES (Advanced Encryption Standard) encryption or the like, to reduceand/or minimize security holes that exist in other wireless protocols.The Thread network may be scalable to connect devices (e.g., 2, 5, 10,20, 50, 100, 150, 200, or more devices) into a single network supportingmultiple hops (e.g., so as to provide communications between deviceswhen one or more nodes of the network is not operating normally). Thenetwork 1000, which may be a Thread network, may provide security at thenetwork and application layers. One or more devices communicativelycoupled to the network 1000 (e.g., controller 1030, remote system 1040,and the like) may store product install codes to ensure only authorizeddevices can join the network 1000. One or more operations andcommunications of network 1000 may use cryptography, such as public-keycryptography.

The devices communicatively coupled to the network 1000 of the smarthome environment disclosed herein may have low power consumption and/orreduced power consumption. That is, devices efficiently communicate towith one another and operate to provide functionality to the user, wherethe devices may have reduced battery size and increased batterylifetimes over conventional devices. The devices may include sleep modesto increase battery life and reduce power requirements. For example,communications between devices coupled to the network 1000 may use thepower-efficient IEEE 802.15.4 MAC/PHY protocol. In embodiments of thedisclosed subject matter, short messaging between devices on the network1000 may conserve bandwidth and power. The routing protocol of thenetwork 1000 may reduce network overhead and latency. The communicationinterfaces of the devices coupled to the smart home environment mayinclude wireless system-on-chips to support the low-power, secure,stable, and/or scalable communications network 1000.

The sensor network shown in FIG. 10A may be an example of a smart homeenvironment. The depicted smart home environment may include astructure, a house, office building, garage, mobile home, or the like.The devices of the smart home environment, such as the sensors 1010 and1020 the controller 1030, and the network 1000 may be integrated into asmart home environment that does not include an entire structure, suchas an apartment, condominium, or office space.

The smart home environment can control and/or be coupled to devicesoutside of the structure. For example, one or more of the sensors 1010and 1020 may be located outside the structure, for example, at one ormore distances from the structure (e.g., sensors 1010 and 1020 may bedisposed outside the structure, at points along a land perimeter onwhich the structure is located, and the like. One or more of the devicesin the smart home environment need not physically be within thestructure. For example, the controller 1030 which may receive input fromthe sensors 1010 and 1020 may be located outside of the structure.

The structure of the smart home environment may include a plurality ofrooms, separated at least partly from each other via walls. The wallscan include interior walls or exterior walls. Each room can furtherinclude a floor and a ceiling. Devices of the smart home environment,such as the sensors 1010 and 1020, may be mounted on, integrated withand/or supported by a wall, floor, or ceiling of the structure.

The smart home environment including the sensor network shown in FIG.10A may include a plurality of devices, including intelligent,multi-sensing, network-connected devices, that can integrate seamlesslywith each other and/or with a central server or a cloud-computing system(e.g., controller 1030 and/or remote system 1040) to providehome-security and smart home features. The smart home environment mayinclude one or more intelligent, multi-sensing, network-connectedthermostats (e.g., “smart thermostats”), one or more intelligent,network-connected, multi-sensing hazard detection units (e.g., “smarthazard detectors”), and one or more intelligent, multi-sensing,network-connected entryway interface devices (e.g., “smart doorbells”).The smart hazard detectors, smart thermostats, and smart doorbells maybe the sensors 1010 and 1020 shown in FIG. 10A.

For example, a smart thermostat may detect ambient climatecharacteristics (e.g., temperature and/or humidity) and may accordinglycontrol an HVAC (heating, ventilating, and air conditioning) system ofthe structure. For example, the ambient climate characteristics may bedetected by sensors 1010 and 1020 shown in FIG. 10A, and the controller1030 may control the HVAC system (not shown) of the structure.

As another example, a smart hazard detector may detect the presence of ahazardous substance or a substance indicative of a hazardous substance(e.g., smoke, fire, or carbon monoxide). For example, smoke, fire,and/or carbon monoxide may be detected by sensors 1010 and 1020 shown inFIG. 10A, and the controller 1030 may control an alarm system to providea visual and/or audible alarm to the user of the smart home environment.

As another example, a smart doorbell may control doorbell functionality,detect a subject's approach to or departure from a location (e.g., anouter door to the structure), and announce a subject's approach ordeparture from the structure via audible and/or visual message that isoutput by a speaker and/or a display coupled to, for example, thecontroller 1030.

In some embodiments, the smart home environment of the sensor networkshown in FIG. 10A may include one or more intelligent, multi-sensing,network-connected wall switches (e.g., “smart wall switches”), one ormore intelligent, multi-sensing, network-connected wall plug interfaces(e.g., “smart wall plugs”). The smart wall switches and/or smart wallplugs may be or include one or more of the sensors 1010 and 1020 shownin FIG. 10A. A smart wall switch may detect ambient lighting conditions,and control a power and/or dim state of one or more lights. For example,a sensor such as sensors 1010 and 1020, may detect ambient lightingconditions, and a device such as the controller 1030 may control thepower to one or more lights (not shown) in the smart home environment.Smart wall switches may also control a power state or speed of a fan,such as a ceiling fan. For example, sensors 1010 and 1020 may detect thepower and/or speed of a fan, and the controller 1030 may adjust thepower and/or speed of the fan, accordingly. Smart wall plugs may controlsupply of power to one or more wall plugs (e.g., such that power is notsupplied to the plug if nobody is detected to be within the smart homeenvironment). For example, one of the smart wall plugs may controlsupply of power to a lamp (not shown).

In embodiments of the disclosed subject matter, a smart home environmentmay include one or more intelligent, multi-sensing, network-connectedentry detectors (e.g., “smart entry detectors”). Such detectors may beor include one or more of the sensors 1010 and 1020 shown in FIG. 10A.The illustrated smart entry detectors (e.g., sensors 1010 and 1020) maybe disposed at one or more windows, doors, and other entry points of thesmart home environment for detecting when a window, door, or other entrypoint is opened, broken, breached, and/or compromised. The smart entrydetectors may generate a corresponding signal to be provided to thecontroller 1030 and/or the remote system 1040 when a window or door isopened, closed, breached, and/or compromised. In some embodiments of thedisclosed subject matter, the alarm system, which may be included withcontroller 1030 and/or coupled to the network 1000 may not arm unlessall smart entry detectors (e.g., sensors 1010 and 1020) indicate thatall doors, windows, entryways, and the like are closed and/or that allsmart entry detectors are armed.

The smart home environment of the sensor network shown in FIG. 10A caninclude one or more intelligent, multi-sensing, network-connecteddoorknobs (e.g., “smart doorknob”). For example, the sensors 1010 and1020 may be coupled to a doorknob of a door (e.g., doorknobs located onexternal doors of the structure of the smart home environment). However,it should be appreciated that smart doorknobs can be provided onexternal and/or internal doors of the smart home environment.

The smart thermostats, the smart hazard detectors, the smart doorbells,the smart wall switches, the smart wall plugs, the smart entrydetectors, the smart doorknobs, the keypads, and other devices of asmart home environment (e.g., as illustrated as sensors 1010 and 1020 ofFIG. 10A) can be communicatively coupled to each other via the network1000, and to the controller 1030 and/or remote system 1040 to providesecurity, safety, and/or comfort for the smart home environment.Alternatively or in addition, each of the devices of the smart homeenvironment may provide data that can be used to determine an occupancyand/or physical status of a premises, as well as data that may be usedto determine an appropriate recipient of a notification, as previouslydisclosed herein.

A user can interact with one or more of the network-connected smartdevices (e.g., via the network 1000). For example, a user cancommunicate with one or more of the network-connected smart devicesusing a computer (e.g., a desktop computer, laptop computer, tablet, orthe like) or other portable electronic device (e.g., a smartphone, atablet, a key FOB, or the like). A webpage or application can beconfigured to receive communications from the user and control the oneor more of the network-connected smart devices based on thecommunications and/or to present information about the device'soperation to the user. For example, the user can view, arm or disarm thesecurity system of the home.

One or more users can control one or more of the network-connected smartdevices in the smart home environment using a network-connected computeror portable electronic device. In some examples, some or all of theusers (e.g., individuals who live in the home) can register their mobiledevice and/or key FOBs with the smart home environment (e.g., with thecontroller 1030). Such registration can be made at a central server(e.g., the controller 1030 and/or the remote system 1040) toauthenticate the user and/or the electronic device as being associatedwith the smart home environment, and to provide permission to the userto use the electronic device to control the network-connected smartdevices and systems of the smart home environment. A user can use theirregistered electronic device to remotely control the network-connectedsmart devices and systems of the smart home environment, such as whenthe occupant is at work or on vacation. The user may also use theirregistered electronic device to control the network-connected smartdevices when the user is located inside the smart home environment.

Alternatively, or in addition to registering electronic devices, thesmart home environment may make inferences about which individuals livein the home (occupants) and are therefore users and which electronicdevices are associated with those individuals. As such, the smart homeenvironment may “learn” who is a user (e.g., an authorized user) andpermit the electronic devices associated with those individuals tocontrol the network-connected smart devices of the smart homeenvironment (e.g., devices communicatively coupled to the network 1000)in some embodiments, including sensors used by or within the smart homeenvironment. Various types of notices and other information may beprovided to users via messages sent to one or more user electronicdevices. For example, the messages can be sent via email, short messageservice (SMS), multimedia messaging service (MMS), unstructuredsupplementary service data (USSD), as well as any other type ofmessaging services and/or communication protocols. As previouslydescribed, such notices may be generated in response to specificdeterminations of the occupancy and/or physical status of a premises, orthey may be sent for other reasons as disclosed herein.

A smart home environment may include communication with devices outsideof the smart home environment but within a proximate geographical rangeof the home. For example, the smart home environment may include anoutdoor lighting system (not shown) that communicates informationthrough the communication network 1000 or directly to a central serveror cloud-computing system (e.g., controller 1030 and/or remote system1040) regarding detected movement and/or presence of people, animals,and any other objects and receives back commands for controlling thelighting accordingly.

The controller 1030 and/or remote system 1040 can control the outdoorlighting system based on information received from the othernetwork-connected smart devices in the smart home environment. Forexample, in the event that any of the network-connected smart devices,such as smart wall plugs located outdoors, detect movement at nighttime,the controller 1030 and/or remote system 1040 can activate the outdoorlighting system and/or other lights in the smart home environment.

In some configurations, a remote system 1040 may aggregate data frommultiple locations, such as multiple buildings, multi-residentbuildings, individual residences within a neighborhood, multipleneighborhoods, and the like. In general, multiple sensor/controllersystems 1050 and 1060 as shown FIG. 10B may provide information to theremote system 1040. The systems 1050 and 1060 may provide data directlyfrom one or more sensors as previously described, or the data may beaggregated and/or analyzed by local controllers such as the controller1030, which then communicates with the remote system 1040. The remotesystem may aggregate and analyze the data from multiple locations, andmay provide aggregate results to each location. For example, the remotesystem 1040 may examine larger regions for common sensor data or trendsin sensor data, and provide information on the identified commonality orenvironmental data trends to each local system 1050 and 1060. Aggregateddata may be used to generate appropriate notices and/or determineappropriate recipients for such notices as disclosed herein. Forexample, the remote system 1040 may determine that the most common userresponse to a notification that a garage door has been left open while asecurity component of the smart home environment is in an armed state,is that the user returns to the premises and closes the garage door.Individual smart home systems and/or controllers as previously disclosedmay receive such data from the remote system and, in response, set adefault action of closing the garage door when the system determinesthat an armed state has been set and the garage door has been left openfor more than a minimum threshold of time. The data provided to theindividual systems may be only aggregate data, i.e., such that noindividual information about any one other smart home environment ortype of smart home environment is provided to any other. As anotherexample, the remote system may receive data from multiple premises in aparticular geographic region, indicating that it is raining in theregion, and that the rain is moving east (based on the times at whichthe data indicating rainfall is received from different premises). Inresponse, the remote system may provide an indication to premisesfurther to the east that rain may be expected. In response,notifications may be provided to occupants of the individual premisesthat rain is expected, that particular windows should be closed, or thelike. In some configurations users may be provided with the option ofreceiving such aggregated data, and/or with the option of providinganonymous data to a remote system for use in such aggregation. In someconfigurations, aggregated data also may be provided as “historical”data as previously disclosed. Such data may be used by a remote systemand/or by individual smart home environments to identify trends, predictphysical statuses of a premises, and the like.

In situations in which the systems discussed here collect personalinformation about users, or may make use of personal information, theusers may be provided with an opportunity to control whether programs orfeatures collect user information (e.g., information about a user'ssocial network, social actions or activities, profession, a user'spreferences, or a user's current location), or to control whether and/orhow to receive content from the content server that may be more relevantto the user. In addition, certain data may be treated in one or moreways before it is stored or used, so that personally identifiableinformation is removed. For example, specific information about a user'sresidence may be treated so that no personally identifiable informationcan be determined for the user, or a user's geographic location may begeneralized where location information is obtained (such as to a city,ZIP code, or state level), so that a particular location of a usercannot be determined. As another example, systems disclosed herein mayallow a user to restrict the information collected by those systems toapplications specific to the user, such as by disabling or limiting theextent to which such information is aggregated or used in analysis withother information from other users. Thus, the user may have control overhow information is collected about the user and used by a system asdisclosed herein.

Embodiments of the presently disclosed subject matter may be implementedin and used with a variety of computing devices. FIG. 11 is an exampleof a computing device 1100 suitable for implementing embodiments of thedisclosed subject matter. For example, the device 1100 may be used toimplement a controller, a device including sensors as disclosed herein,or the like. Alternatively or in addition, the device 1100 may be, forexample, a desktop or laptop computer, or a mobile computing device suchas a smart phone, tablet, or the like. The device 1100 may include a bus1110 which interconnects major components of the computer 1100, such asa central processor 1140, a memory 1170 such as Random Access Memory(RAM), Read Only Memory (ROM), flash RAM, or the like, a user display1120 such as a display screen, a user input interface 1160, which mayinclude one or more controllers and associated user input devices suchas a keyboard, mouse, touch screen, and the like, a fixed storage 1130such as a hard drive, flash storage, and the like, a removable mediacomponent 1150 operative to control and receive an optical disk, flashdrive, and the like, and a network interface 1190 operable tocommunicate with one or more remote devices via a suitable networkconnection.

The bus 1110 allows data communication between the central processor1140 and one or more memory components 1150 and 1170, which may includeRAM, ROM, and other memory, as previously noted. Applications residentwith the computer 1100 are generally stored on and accessed via acomputer readable storage medium.

The fixed storage 1130 may be integral with the computer 1100 or may beseparate and accessed through other interfaces. The network interface1190 may provide a direct connection to a remote server via a wired orwireless connection. The network interface 1190 may provide suchconnection using any suitable technique and protocol as will be readilyunderstood by one of skill in the art, including digital cellulartelephone, Wi-Fi, Bluetooth®, near-field, and the like. For example, thenetwork interface 1190 may allow the device to communicate with othercomputers via one or more local, wide-area, or other communicationnetworks, as described in further detail herein.

FIG. 12 shows an example network arrangement according to an embodimentof the disclosed subject matter. One or more devices 1210 and 1211, suchas local computers, smart phones, tablet computing devices, and the likemay connect to other devices via one or more networks 1200. Each devicemay be a computing device as previously described. The network may be alocal network, wide-area network, the Internet, or any other suitablecommunication network or networks, and may be implemented on anysuitable platform including wired and/or wireless networks. The devicesmay communicate with one or more remote devices, such as servers 1212and/or databases 1213. The remote devices may be directly accessible bythe devices 1210 and 1211, or one or more other devices may provideintermediary access such as where a server 1212 provides access toresources stored in a database 1213. The devices 1210 and 1211 also mayaccess remote platforms 1214 or services provided by remote platforms1214 such as cloud computing arrangements and services. The remoteplatform 1214 may include one or more servers 1215 and/or databases1216.

Various embodiments of the presently disclosed subject matter mayinclude or be embodied in the form of computer-implemented processes andapparatuses for practicing those processes. Embodiments also may beembodied in the form of a computer program product having computerprogram code containing instructions embodied in non-transitory and/ortangible media, such as hard drives, USB (universal serial bus) drives,or any other machine readable storage medium, such that when thecomputer program code is loaded into and executed by a computer, thecomputer becomes an apparatus for practicing embodiments of thedisclosed subject matter. When implemented on a general-purposemicroprocessor, the computer program code may configure themicroprocessor to become a special-purpose device, such as by creationof specific logic circuits as specified by the instructions.

Embodiments may be implemented using hardware that may include aprocessor, such as a general purpose microprocessor and/or anApplication Specific Integrated Circuit (ASIC) that embodies all or partof the techniques according to embodiments of the disclosed subjectmatter in hardware and/or firmware. The processor may be coupled tomemory, such as RAM, ROM, flash memory, a hard disk or any other devicecapable of storing electronic information. The memory may storeinstructions adapted to be executed by the processor to perform thetechniques according to embodiments of the disclosed subject matter.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit embodiments of the disclosed subject matter to the precise formsdisclosed. Many modifications and variations are possible in view of theabove teachings. The embodiments were chosen and described in order toexplain the principles of embodiments of the disclosed subject matterand their practical applications, to thereby enable others skilled inthe art to utilize those embodiments as well as various embodiments withvarious modifications as may be suited to the particular usecontemplated.

1. A method for determining an occurrence of a sleep behavior ordisorder, the method comprising: receiving, from a radiation capturecomponent of an electronic device, a first signal corresponding to asequence of radiation reflected from a subject; filtering the firstsignal by reducing at least one energy peak of the reflected sequence ofradiation to produce a second signal; determining a heart rate of thesubject based on the second signal; determining an occurrence of a sleepbehavior or disorder based on the heart rate; and in response to theoccurrence of the sleep behavior or disorder, causing a notice about theoccurrence of the sleep behavior or disorder to be provided.
 2. Themethod of claim 1, further comprising: receiving, from a sensor,information about a temperature in vicinity of the subject.
 3. Themethod of claim 2, wherein the first signal is filtered based on thetemperature.
 4. The method of claim 1, wherein the notice is provided toa mobile device of a user separate from the subject.
 5. The method ofclaim 4, wherein the subject is a juvenile and the user is an adult. 6.The method of claim 4, further comprising providing an alert to ahealthcare provider or an emergency responder.
 7. The method of claim 1,wherein the notice is provided via the electronic device.
 8. The methodof claim 1, further comprising: determining a sleep stage of the subjectbased upon the first signal; and storing a record of the first signaland the determined sleep behavior or disorder in a sleep profile of thesubject.
 9. The method of claim 8, further comprising: storing a recordof the heart rate and/or the second signal in the sleep profile.
 10. Themethod of claim 1, wherein the notice is provided to a home monitoringhub.
 11. A system for determining an occurrence of a sleep disorder, thesystem comprising: a memory configured to store a first signal and asecond signal; and a processor configured to: receive, from a radiationcapture component of an electronic device, a first signal correspondingto a sequence of radiation reflected from a subject; filter the firstsignal by reducing at least one energy peak of the reflected sequence ofradiation to produce a second signal; determine a heart rate of thesubject based on the second signal; determine an occurrence of a sleepbehavior or disorder based on the heart rate; and in response to theoccurrence of the sleep behavior or disorder, cause a notice about theoccurrence of the sleep behavior or disorder to be provided.
 12. Thesystem of claim 11, the processor further configured to: receive, from asensor, information about a temperature in vicinity of the subject. 13.The system of claim 12, wherein the first signal is filtered based onthe temperature.
 14. The system of claim 12, further comprising thesensor.
 15. The system of claim 11, wherein the notice is provided to amobile device of a user separate from the subject.
 16. The system ofclaim 15, the processor further configured to provide an alert to ahealthcare provider or an emergency responder.
 17. The system of claim11, wherein the processor is configured to provide the notice via theelectronic device.
 18. The system of claim 11, the processor furtherconfigured to: determine a sleep stage of the subject based upon thefirst signal; and store a record of the first signal and the determinedsleep behavior or disorder in a sleep profile of the subject.
 19. Thesystem of claim 18, wherein the memory further stores the sleep profileof the subject.
 20. The system of claim 11, the processor furtherconfigured to provide the notice to a home monitoring hub incommunication with a device comprising the processor.